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AdSetfaod of and S j^tem to $et an ouiput quality of a 
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We consider a scalable media processing algonthm diat has been allocated a fixed 
CPUbudget, Le., it has been allocated afixed shace of the CPU power, on which it 
10 Iiastogqtby. 

A a qi^ality level control strategy is presented for a media processing algorithm v^^ith a 
fixed budget, based on Moikov decision procossos, that is used to 
1. minimize the number of deadline misses, 
15 2- nmaixntee the nmnbrar of quality level changes^ and 
3. maximize the quality level of processing. 

To do so» it keeps track of a so-caned rela^dve progress and the previously used quality r 
level. Sa addition to the method presented, we may apply so-called budget scaling, 
that €:an also handle long'*tetm load fluctuations* 

20 

in the above method the Markov decision process is solved ofif-line, based on a 

number of typical video sequmcas. Next, the resulting control ^rategy is loaded into c 
the nm-tEme system (Crg. in Uie form of a table), to do the actual contzoL This table 
xeonains fixed dnrmg pzocessing. A drawback of this method is diat the video 
25 sequences that are encountered doting run time may be difibrentfiom the ones, used to 
conpu^ tite control strategy. As aresult, the ccmtcol may be sub-optunaL 
FUrthennore» a drawback is that the system depends on a table diat has to be loaded, 
andwhlchhas to be confuted a£f-Iine Ci.e^ the system is not self-contained). 

30 To ovexcome the above problems, we propose a self-learning control strategy. Yor 
whicdi we basically choose an on-line vanant of the Markov dedsion process (it is 
based on leinfomiment learning or mote specifically, on so-called Q-leaming)* In 
ihis way> an optimal control strategy is created ou'^line, tuned to the video sequences 
that are actually encounimed. No control table needs to be loaded, but it is constructed 
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fiUBmally by the method itself. To make the Q-leaming approach more effident. we 
developed a method that extr^olates the effect of taldng an action in a csrtain'staie to 
aU possible actions in an possible states, fa this way, the Q-leaming approach leams 
drastically faster than usual. 
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t we di^;us cbe teclmii]ues dxat uodexiie tba s^-leanms quality ccmtiol stxategy^ 
■■B^ Tte ccmtml stzat^ is based on Reii£fi:>icecnent Leacaisg and i^i 
partbnilar 0X1 tecbniqi^ SARSA ai^ 

Kssy in Rdnfion^nent L^nxiiig are 50-<^lled ^£ate»acti02i)-Yalaes. We define the stais of ibd 
appliqatioii dt a mil^toiie 33 tbe oombmation of ihe scaled bud^t at tb9 zoileston^ tfae rdative 
{nt>SEes$ at Ite ixiil^ticme^ aiul tbe qual^ Per 
state .f and qpall^ action a» a (3tate,acdon>va^ This value ceAecisfiid expected 

jfiwaid in die long ion when takmg quality action a. starting from state At each mil^toser if 
fbc ^pl&^ttoB is in a state the gualiQr acdon a is cbosean that baa die largest vahie Q{s;a), A 
problem is ttrnt we only can work with (^soraca states, while both tbe scaled bud^ and flie lelative 
progress aro conliauons variables. To solve ttns problem* we first define a discrete set of scaled 
budget points and a discrete set of relative progress pwits. Next, the idea is ihax only for the 
disoete gd^oinis in the 3-dimensioiiaI (sealed budget rdladve piogreiss» previous qnali^) space^ 
weteeptxadcof tfae(stato»BCtion>vahie5. lbcqpproidniatBthB($ta!ie.aciion>vaIuefarastacetfaa^ 
does not cOEzespond to a gFidpomt; we use lin^rimerpclation. 

iiKU after proce^g a unit of work, (he (state^actionVvalne Q(s{a) is updated that corr^ponds 
to the state s pnor to processing the unit of wotlc The quality action a is the qualitjr that is dioaen 
to process the mdt of work, Le., the qualify acdon a fbr wMch Q{s;a) was UgbesL The update 
is done by usnig the direct revenue of action a in stats s and the (stat^acdon>vah^ Qisfic/) that 
conesponds to tibe states' that ihe ap^dicadonte (^laliQracdon 
^ Kgaht Is Che quality acdon&r which Q(^;ol) is U^tiest 

Nonnally, in KL only (he value Q(^a) is updated. However* we talse a dlfferemt approach by 
not iqidating only one ^tate,acdon)*value, but instead by updatfrig die (stai}e,action)-values at all 
gdd^ints and for all possible quality actioa& We do das as follows. After processing aunit of 
work in a certain quality, we ficist estimate the processing time of die unit of work in each other 
quality a (see ste^p 6 of the third algodtlmi) , Nesct* fi>r an gddpomts s, we determine the eSet^ of 
taldng all positfUB actions ain state hQ,^ we detennhie (he lesnldi^ state if^ and w use this in 
the oadidonal way to updaie Q{sid), We can do this, as we basical^ only need the (esiimaiBdO 
processing dme fbr dda ^ee steps 7-11 of tiie third algoridnn). 

For details* we xe^lEbr to the following pseudo^o^ ]h the code, iS^dp stands fbr scaled 

budg^pdto, rpp stands fbrtdadve ptogo^ poistr zoipq stands fbrpceviousliy used qualify. 9m 



till 



iflMtaiiiiPMPii 



pii 



dial 



1* ]n&&ilizethenmningcQmp!GGation&c(w 

2. £baf all sfales (sbp^rpp^pq) 

3, fiur all qualitjr actions 4r 

4v initialize the (^tetacdon)-vaIne 

Qisbp,rp,pq;q)ir^O 
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Algoriflnnge/ dgc&&)n qimUty 
Input; relative progne^ r/> 
Inpufc previously used gaal% 
Output; decision qualilt^ 

li conapatB tfte scaled biu^i 

3. 'tol*o.,aiailyrf,i,lte,,„fli,y.^,art,>,,,,,p^ 



beford processing the last unit of worib wKiwaa mat we tsad 

2. i3seprand^tot5?datetbeniOTmgconqpIi^^ 



3. compuie the scaled budget 



5. fiirallqiialily actions ^ 

^^^feg°P^BSStng thne of the lastunit of wod:&r<iuaaty ^ 

and having a iionnali7 ^ piOeMdqg thne ept/ald»:f 
obsarve both ibeiesniiifleievfflwe w amd the resultnig relative progress rp 

possiblB qualily acdons 5* su 
fl' is die i&^sast vOm (hwi^^rp^ g-,^ 
11. the Cstatwctio5)-valM i2(^p,rpp,pq,g) using w and 

Refemices 
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Adaptrve QoS Control for Real-Time Video ProcessS&g 
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Prof. Holstlaan 4, 5656 AA Eindhoven. The Netiierlands 
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Abstract 

Video pr&cessing in scfiwart is diartu^erLoed by highly 
flucmtini, conxanf-dependen^pmcas^ rimej^ as well as 
strict r^l-timeKqiUrsments, Jntfworsr^casB resource aUo" 
emit>n is net cost^ffec^a. We present an approach hosed 
on asyndhfonous^ scalable (or mtprecise^ processing and 
QqS adcptadarr^ which aUaws close^tO'mmge resource a£^ 
location. We propose two stmegiesforconfrolling the QoS- 
level durk^ rmhfkne, one static, tmdone dynamic. We en- 
hanced baA strategies ifM a can^ensatjonjbr structund 
(tunhstoduistie) loadfitematfon. I^nalfy we valUkae our 
approach hy means of a stmatadm experiments 



1 latEodiiclion 

Oomsomer tennxnals. svcb as seMop boxes sad digical 
TV^^Bts, con^tly dedUseSed Ziardwarft compon^itift 
tx> pzoce^s yideo. In tiiaforeiseeabl&fbtiire^iaogra^ 
haxdwaxeis expec^ted to takeover. Frogz»niniablebard«m6 
las tiie ody^umge of beins applicdidd to fmr^ codiag scan- 
dacds, azid it allows the user to enhaoce the finicfipniiUty 
of consumer tenaiiiaL CSooseqiifliitly, video proce^Jog 
wlQ mova fiom baidwaie ID softwate as soon as Mooze's 
.lawpennits, 

:&gh ^iualicy video lias fitdn^ent jitter zeqpir^oaente; 
deadjlne jnisse? axa ccmsidftrftd exrois that affect thft mean 
time becveen ^dtsre. Bv^ :&ainfi (or field in inteiiaced 
video) has to bet ovqibible when Ihe nwIesBr needs it for 
timely display. 

Sofinrane video processing is diazactedzed by highly 
dactuadng zesonzice feqnirmeiiits, dne to contient d&gtv^ 
denczes, For egcBxcple, FigtUe 1 shO«^ tbft decoding dmes 
for a. sfi^encfr of MPBGMl fiesimes on a TdMedia 1300 
(jZiMBz) piocestsor P], In the figoie utre cm iden^ both 
ffttiQiDTal lo^ flwrttiifttiftrtft^ as» foe instance, within frames 
l^XOQ and lOI-^lO, and stcuctanil load fiufitoatioiis. Strac^ 
tural load Uvsstaasions are, amongst others^ caused by the 
vazying complexi^ of video scenes. Typicany» theze is a 



ktfge gap betve^ the wof6&.case and annmse^case decode 
ing times. 



99 
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Piguro 1 . The decoding tf mas tor a seiqiiBnce 
of MPE&2 frames, showing both tomporai 
and atruetural load fluctuations 

tliisl6adatoadOeauna.Oi)th60iifilmnd,woz^-<^ V 
source allocation is not cost-^effective, sinco con$um6rTgr» / 
minals axe heavily xesouzoe-constrsined doe to high pres-* 
sma on cost On the ccberhaad, assigning resoujces closer 
to Ihe avezsise^case 1o9d dtnation may lead to deadline 
misses and/or diBtQd> ^Ucationsin otberpazfts of the sys* 
tern. 

We address this dilonma as follows. Kicst, we pmvido 
temporal isolation between this task and other tztsks in the 
system, by aHocaiing a guaranteed periodic CFtJ budget to 
d^taskr IhisxemovesintBEtosnceizomotfawtB^anai- 
Iqw9 us to amcentcatt on thi$ cnft ta^ ^y» Second, wo 
even one the CPtJload by asynofaM>nou3 video pxocessing 
and woddxtg ahead; aftorprocessiosafiame the task imme* 
diately can sf^piocessmg the nexc&mneft widiont wftituig 
for the deadline. Iho extent to whidi tvoiSdngr ahead can 
be ^plied is detensh^ed by latency and bufE^xing lindtBr- 
dons. Tbucd, we use scalable video algozitbms» which allow 
a tntde-o£f between tbepxocessingthne and die output qual- 
iiy t3]. llheaa scalable video algoiithins apply ogppioKimato 
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piocesfiiDg [6] to the video domaiii. TOiii thfise ingittdifiaB 
in ptoe. we opfimize tibe Qualify of &ndce (QoS) of the 
tasfc» by coniEOjlin^ the proaessing quality for each iudivid- 
aal fiame. hi fliis \^re describe a aumber of «w«rol al- 
gon'dims tbst optimize QoS by baJandDtg deadline misses, 
processing quality, and changes in the pmcessing quality. 

2 QoS BM Iteunewark 

The effort desctibedm tfaisp^eris embedded in almger 
effort, which defines ^nd bunds 51 ftamewoifcftirQoS re- 
soaitse manageioeni; caned QoS RM 7J. M^tmiaining 
the properties of zobusqiess imd oos^effietiveatefis oie ma- 
^requicemeniff dioc axe imposed on this fiamewoxfc ITje 
ibmeworfcconsists of two roaiopans, a Resomca Manager 
(RM) and a QnaHty Managftr (QM). taid is based on two 
main concepts, qi«Uly level and resource budget 

The QM nssigos coniB^-grained quality levels and re 
sonjre btafefits to so-<aUed Rpsomce Consmning Entities 
atC3Es), in a way that is yery simUnr to the approach de- 
scribed in [4]. A com^e-srahifid quality level consisra of a 
number of fiae-^g^ained quality levels, which roughly pn>- 
Yuie the same course-grmn pn?cessing quaUty. Durina the 
steady state betwean two QM deoiitions, tjid RM provides 
guaranteed and esforoed xesouceo bud^ts. In a way that 
IS very similar to die approach desiaibBd m C5J. TOes^rt- 
fiOOTcebudgeteprovidsatfirmalpjocesgarpJ; runnioeon a 
ftaotUai a of a proi^sfior wiii clock fiequency J» is ahnost 
tb& samer as running on a private processor with clock fre- 
qnenoyoci^C^], i?^RCBcoiiiioIIerwiaiin the RCE makes 
sum thar the Rcas produces an acceptable result giveo the 
resource budget provided by the RM» l>y conCtoUing the 
fine-grained quality leveL 

The woilcpresftntedin ftisp^er is aimed at RCEs in the 
context of a QoS m fiainewoifc hut ii can also be Mplied 
QUtside dOsconnsct la thopiesenceof a QM, the RCB conr 
troDer can request a budget increase in case of a smkcnual 
load increase, A stand-alone s^licalioncaanot do tbiSr and 
-mustdiereforebeableio address stnictnmlloadfluotuafions 
as wiftU as temporal load fiuetuaiions. 

3 Basjlc Appiroach 



We consider a single asynclnsooaous scalable video pro- 
cess tasic wiQi an associated csontrollec At each mile- 
SKme, when a fiame is completed* and a next fiame is about 
to be processed, the controller decides vrfiat to do next 
Jx, may decide to sk^ a feme, because it has previously 
missed a deadline, but normally it simply selects a quality 
level fi>r the naxtfiramo. EaiAfiname must be oompleted be- 
fore the corresponding deadline. Suooessivo deadlines ai9 
sctictQrpeilodio. widi agiyenpetiod />. Theprocfissfngtime 
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for afiame dependsonbothihodiaseaqualtolevelandftft 
content of the fiame. During each deadline period, the task 
can usa aguamnteedpioeessiiig-tirae budget ib, 

^""^ ^ renderer Coutside the 

imphes a QoS lednction for the stream. IhectmrroUerde. 

iBCts a possible deadlizidrUissafiDeFi - - ^ 

at die milestone, and accounts for i 



"^aI^SIT^"*^ aupgcijuem aeaaimes by one pa- 
nod^ ^,^&asdetectedadcadImemls^,fl,econtroltod^ 

fiian asyhchronons approach, iris important to know the 
^MvapnDgr^atamilestoaecomp^ 
ingdeadlin^ because a larger progress leads 10 a low^rfsk 
^mi5«ngd,anaxcdeadUne, Frograsa, measured^a^ 
Stone, IS ttototal amount of guarameed budget left 
^ne^cmdHng deadline, dMded by the periodic budget & 

^^fteprogress.thecomroUermoLom^^^ 
aUoc^to tibemsk. The progi«ss Is bounded from above 
due 10 fimte bufifer sizes, and is always non-negatrv^ fa^ 
^ofdieshifting Of deadlmos incase of deaiflS^^ 
JSGtt tim aborting the processing at adeadlino miss woold 
not inake sense, U would only decrease the ntogt^ and 
^ iii^e the risk of anotherdeadZine miS/Kwr 
ft IS harder to abort a fiame than to finish it "^"^^ 

3.1 Frobl&nStsOBmmt 

eontmllar has to choose tijo qual- 
ity level at which the upcoming feme is processed. liL 
problem we consfd^ is ro choose this qualiiy level In sneh 

awayihatthafollowingihreeobjectiv^m^^ 
CMsedeadHn 

b& ou^utqualror, flames should be ptocessefl aitheSi- 

^f^f^^^'y^^ Ohinlftomanberandsi^of 
quahty kvel changes sbotild be as low aa possible, because 
(b%ger) ^^inflte quality level may ie*mlt ^ (bX) 
Wvableartift^ Clearly, meeting aflflireeobjeoWS 
Je CTie time vs impossible, so we have to find i opiimal 



3^ Solution Approach 

To ad&ess dre above problem, we have mod^ it as a 
Markov Deapion^^^ CMDP) ffij and as a Reinfoice- 
m^c txjarmng flRL) pn,blem [IQJ, Both methods are^ 
lated, mdcondderaq^ tfaqtiscbaraijterizedby a set of 
fitates, aset of possibje actions foreacfa TOie, and numerical 
S^nrin'^T'^?'*^^ At discrete moments in time, 
action. Theacuonimtiaiesaaiaretransitioa, whichdepends 
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on the aciioa iiseif and on the systeiu's dynai^^ 
Combat is fakes a teveim&is olndned. l%e ffol of boiii 
appxoadifisiis to imudixiizetfae avemgercvenue over aQ state 
transitions. 

For OCT problem, tiie state of iht system is detenniiiEd 
by ^ task's progress, and by iis quoUry level, hottk at tfte 
ctment milestone. lite fiction is the qnality level for the 
next ftamer A levEimo is composed of a (high) penalty on 
tbp eocpeofi^nttmbar of deadline mis^ be^breieai^iing the 
nfixc mOe^ne^ a reward fbrohoosing a pazfSculareiaaliQr 
level at die ndlesnma^ and a pettalty fot dioosing a qunHty 
level di£B»xtticfiom die previoiu^ used one. lodds way, we 
fo^neediedneeprofalnnof^fletrves. theohcainedxefeaBe 
can be viewed as en objecdve Qo5 measme. 

Hie MDP is solved offline for a paidetdar proces^og 
budget, ond leqnixes pxe-detennoied probab]litie& of staie 
transitions* Ihe restilt is given by a sts^c policy, wbic{i 
is a sec of fitateroction p^iss. Uns poU^ is appfied dur- 
ing nm"^e to choose (he qtialiiy level atmilestones, ^ven 
the statei Cleazly» the budget applied for processxng should 
match the budget for whidi ^epolicy Is delved. We detiote 
this coi^I strategy by ^SDP. Bor znoie det&ils, we refer 10 
[11]- 

The RL approach can be considered as an on^Iina variant 
of tihe MDR M ia« cozdrca scEafiftgy scaxts 131^ no loiQwl^ 
edge at all, and has co le&m optimal behavior from the ejc- 
petienee it gains dmhig nm-ibne. this is achieved by con- 
structing and mdamlzzing a dymuxdc polu^, Ibmaicb am- 
specific problem, we applied the SARSA dgodthm [10]. 
Tias algodxhm is based on leammg state-action values, 
which Climate how good it is to ohoose a pazticplar action 
in a given state. We took advantage of owe ^eclfic prob* 
lem to mcdiQ; SARSA, This xnodificatiDn enables us co op- 
data an stats-action values instead of Just one at milestones, 
which speeds vp convexgeoee significantiy. The stat&action 
valuer are E^plied to choose the qnalicy level for the next 
fiame. Qivea die state, the qaaHQr level action) yields^ 
the largest statfi-accbn value is chosen. 

4 Stractnral Load Flnctn^oiis 

The MDP 0nd control stratejgxes asstmie sncces^ve 
pvDcesfilngdmestobeindepBodencof eachothet: Thisis 
dieoaseforteii](P0ialload£luctaatlon5, butnotforstmctuial 
lo^idfinchiatioiis. ht tliis section we compeosateMDP and 
RLSorthisshotcooming* lb tmck the stmctox^ load dur- 
ing nm-ismB, we first compare the actual processing time 
flpyofacQmpleBed frame x^th the expected processing time 
ept for a frame at the applied quality leveL We express ttie 
relation as a complication factor qf, Lg^ cf=: apt /ept. Hie 
^rvalues axe deirved<^-Iine^ We filter out temporalload 
fluctuations by computhig a tunning average rqf over the 
succes^e encount&ced Rvalues. If n^deviates hem one. 



diBB it fiaels to die cask as if the processing bu(^er deviates 
from dift available budget ib* In case of ncf a lJ2,afaudget 
b^3Q ms would feel as abndgctof only 2jf ms, and in case 
of nfss 0.8, aai same budget wonldfeel qs a budget of 37*5 
ms. We th^efoie define a scaled Imdget as b/rtf^ During 
cunrtime^ wecsTmpms die scaled budget at each milestone. 

Given die scaled budgets, we nu»Sfied Che strategies 
MDP and RL OS follows. Xn the MDP qtpioa;^, we deiive 
policies fbr a selected set of scaled budgets* Dming nm- 
cinie, or eadi milestone^ we ^^prokfinam apoEey fbr die 
current scaled budget by inteipolanng between two derived 
policies* fii the RL-probl^ we Include die scaled budget 
Greedy into die state space, ^doaotetheseadapted ver- 
sions of MDP aoadKLl^MDP'^ and RL*»xBspectiveIy. F)or 
details on MDP*, we refer to tlSQ. 

S PerfiDrmaiice Results 

lb test dsa <fi£S3teat control stiat^e$» we have mn a 
simulation expedment la diis ejqmctmen^ we &st mear> 
sured the per-frame pioces^ng times for decoding a com- 
plete movie, oonsisiing of 14J ^3 games, using a scsdable 
MFBGw2 decode with four quality levels, in or- 

der of increasing quality, ronidngon a l^JMedia platfionn 
[91. We produced four traces, one for each quality leveL 
With a frame rate of 25 ftames/sec, we measured a worst- 
case processing time of 77.4 ms and an average-case pro« 
cessing tune of 26^8 ms, for the highest quality level, ^3. 
We used these teaces to derive scate^transitlQn luobabiHties 
forthah^P, to deteimina^/ir values fbr budget scaling and 
to stmalate dieprocessiiis under die differentcontxol stzate- 
gles. In [12] we de$enbft an expeiimait in wbidi MDP* Ift 
filled on video sequences other than cba one used to coxh-- 
pnte tha state-teassldon probabilities. The performance of 
MDP* in tiie curEvmt expeifiment is roughly the same. 

We set the upper bound on progress at 2, which implies 
that that the decoder can wodc ahead Ir/ at most cwo dead- 
line peuods. Based on discussions wiA e3q)exts in the video 
domain, we de&ed the revenues as follows. For each dead* 
linemiss wegaveapenalty of XO,C0D. For choosing quality 
level ^Of • gave rewards of 4^ 6, S> and 10, leapec- 

txvely« For increasing die quallcy by one level we gave a 
penalty of 10, for two levels we gave a penalty of 100, and 
for three levels we gave apeaally of 1000. For decreasing 
die quality level diese penpldes were multtpliedby five. 

Pbr ^^^rencooatml snstegtes, MDP, MDP* RL*» Q3, 
we sfnudamd procesdi^ die mowia at tfifibroa budgets; 
Q3 denoiesdie straigfacforwazdsttaie^ of always choosing 
quality level 93* Hgure 2 shows die average nevenue diat 
we measured in these simnladons, as a fbncdon of die budi 
get Ushig dynamic progranunhig we coniputed the npper 
bound that can be obtained on the average revenue, which 
we denote by UB. This upper bound can not be met by 
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axiy ccmcrol strategy, because it requites conplete Imowl- 
fidgeonaDfiimrepKHMSstof fSnxes. SMegiesMDP*aad 
RL* pctfonsi dose ro oprimam, and their giaph^ rou^Uy 
overtop. Scrategy MDP ptdoims sigmfjcantly worse, and 
snattsgy Q3 perfbtais pooriy, From this result tve concludfi 
am quality control mid budget scaling aie useM- It is inter- 
esting ficom a practical point of vie?w to compare the budgets 
required to ocbififva a certain QoS value (sveage teveaue). 
!?brejcaji^le, to aciiifiEve an QoS value of a,MDP* andRL* 
lequiie a budget of 26.9 ms, MDP requires a budget of 30.0 
ms. and Q3 leqnixes a bndtget of 3 3 .7 ms. 




Figure 2. TTio average revonue for different 
control strategies^ asafunetion of the budget. 

Next, HgUiB 3 shoxQS tbe avmage number of deadlines 
that were missed pei? ftame. Tie graphs for UB, MDP, 
MDP* and m:* overlap, and Q3 perfonns sigoifieaiifly 
At a budget Of 26,9 nw, MDP* andftL* only &ce 
ooeaslonal deadline misses, vilifle on avrnge cre^ 
one d^adliuB ndss Bveiy 25 fiames, vfaidi once per sec- 
end. GleaAy, sndhanumberof deadline misses is unaccept- 
able in a consumartenninaL 




2S 30 
tradgeiOrar) 



Figure 3. The average number of deadline 
mfssee perfmme for different conttol strate- 
gies, as a function of the budget. 



* 22,04.2003 
6 Conduaom and Qngoii^ Work 

We bave pr^emed an appioadi lo handle an average- 
case xesonrce allooadon, assigned to a software video pro^ 
ce^ ca^ that id cfaaiactBris»d by a highly fluctuaimg 
w«Woad and strict real-time requirements Our approach 
is based on asynchronous scalable processing, whic* allows 
a trade-off between die resouroe usage and an overall QoS 
level. l>eriving proper QoS p^iametETs to tiade^ dead- 
line misses andprnocessingqualiiy is atask of video don^ ■ 
experts. We inrroduced two sirategieg |o oonttol tbe QoS 
level during ran-timB. Ilie MDP ^rategy ia based on off- 
^ opnmissaiiaD, whereas iht RL strategy has to leam op- 
tinial behayiorftommn-time experience. We adapted AIDP 
and to handle stmctnnd load ductuadons. TTjetwoiB- 
ailtittgsttttegies, MDP* and proved to perform close 
toc^t^^ m a slmoladon of processing an entire movie 
CJtven an avferage-oaso resource allocation, MDP* and RL* 
met almost all deadlines, Ourcuiientworfccoosistsofafbp. 
ffifir fawestigation imo the rl approach, and inttlemendi^ 
the snatBBies in ji «Htti-Uf^ (system- 
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1 Ihtroducdon 



is dne to the vatyiftg size and coMplexiBr af media ddtt. <!*^rtnrf^ * s«oiscaofly over tune. This 
ing application a^sssing iSSS S^S^StSS^S''''^^"^^^*""'^^^ 

mTsaesioarotastcoiBiLerSSipr "''^ *»™°*» «» 'a*fl«^ wWch con- 



LI BdatedWork 
1»2 Orgaxdzfation 



2 PtoWem Statement and Pceliminari^ 



■lWflBSMn*ta^I>ettp4lli^^yfllndeibJr^!BVftwogh 
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^& sndcDfi^d^dtyof prQce^g. Hsnc^ tbe time reqmied to pxoce^ aunit of wofkis consirifined to be 
scDClmsticral^distdtaHBd, dependent on both At unitGfwnrk type and die quality level. 
'IHie moment m at ^bidb dm pioces^g of a omt of woxic b fnished, is catkd e mite$cone. Rir 
eadi nflestone chere i& a deadKnfiy and <(fari1iftfis ate assumed CD stdcgy perfocBc in dmeu A deadline 
cotr^ponds to a moment in time at wbtch some excemal cries to outact a pxoeeftsed md t of wod^ 
from die ouqmtliafiren tJoiCsGf woricamlcoaaespondingniil^tones and deadlines axe numbered 1,2,..., 
For cony^enoe^ we consider the momentin dma ifs^beforeproc^sfni^ die flist unit of wozk also to be a 
milestonet nomber&d Qy al ifaoi^gh it has no oog^ponding deadBnc 

In eacb deadCne pedodP, the ^licadon is guaranteed a&ed budget b fbrprocessing. We define reladve 
j^ic^^Tmatamiliffitaneas dietDtalQnumu:of diedeadlmeo^tlie milesioner divided 

die peidodic b&dsst If me salaiive piogCe&$ p at a milesione drops below zeto, so it was s^ero or larger 
attbepxevioos milestone, then the ^pHcadon has epwtmtered f-^pl deadHne misses snio» the pxevious 
milestone. Tb deal with deadline mis^p we conaklear vm wo^ pti^esvfng deadline miss ^ciproacbeSr 
One dppztsaeb is lOuse dseompntimmeditoltr i;iKm <»eation» i^e^ at tjiemilestoneat wbioh adeadlntemiss 
is detected. lids lesubs in an adapted relative pirogress of Oat the mHestone. deadHnes for dll n^ 
nnits of woik are pos^oned^y time tmits^ We tef^ to this ^toach as the wnseryative deadline miss 
approach* Note that this aiqiaoa^ presumes diat deadlines oanbe shifted In time at a fti^gmnolaiity dian 
the deadline period. 

Mother ^roacb is to use the ouqnit at ilie fhsc next deadline^ This results in an ad^tsd reladve progress 
of p ppl at die milestone. The deadlines tat all next units of won^am posQponedby f -q\P time units. 
'W^rcfinr to this approach as fha^ft^pih^ deadline miss approach. 

Both deadline miss approach^ xesuh in a non^-negative ad^ted relative progress at ndlestou^* Hietefom, 
a relmive progress of 0 can be considei^ as a lower bound on xelativa progress. Furdier, we assume that 
die ouqruc buffer has a fitite siae^ Iheiefore we also have an upper bound on rdathre progress, deootedl^ 

Praia- 

At eadi milestone^ aqnaliiy level is to bo diosen at which tbene^ctnnit of work is processed. The problem 
we consider is todux)se diis quality level in such a way diatsucdi diat the following three otycctives are met: 
Birsi» the qu^Uiy lev^ at which units of work are processed should be as high as- possible. Secondly, die 
nun^of deaiiUnemis^ should be as low as possible, becansedeadfinemissesi^t la s&vM&xt&EM&'m 
me output ThizdIEy, the number of quality level diange^ should also be as low as possible^ because qualiQr 
level changes also re^t in arte&cis. A Umitmg condition is dm a quality level control strat^runs on die 
same processor ^ the^plicadon, andtherefbre should take little compmaiional resources. 



3 Markov PoKdesi 

laiUs secdon, wo tnodel me problem of Seedon 2 as a finite Mea^ decision process CMDI^. A finite 
Mk±ov decision process desonbes a system that is charaetecized by a finite Set of states, a finite set of 
dod^ons for each state, probabiHties for $tsa» tiansiiions which dep^d on the decision lakjen in a stBte, and 
revues for taking a decision in a state. Solving the Matkov deepen pnxsess results in an optimal M^rfcov 
policy. We useMazkov policies to construct control stcategtes. 



States. Ateachinilesfone weconidderdiestateof ttioapplicalion. Hds state is natniallty given 1^ die 
rdafiveprogress at the mflestone. Because wenecd a fioite sat of states in die yssniiioy deoisiati process« we 
di$e£^i2erelative progress into afiniie set of equti-^d progress intervals between 0 and pta»L^ FUdher, to 
be able to compute revemtes aad ttansidon (wbabiiities later on, we estend the state with botfi the qualiiy 
level at whi^ the previous unit of work waa processed and die type of the n cxt unit of work. 

SecSsians. At e^h milestone a quality level is chosen at which the nest unit of work is processed* These- 
forey we model the sec of denistcns diat oan be t^kan hi a state as the sat of quali^ levels at wSdch tHe 
ai^lieaiion can run. 



3^1 Modal 



2 
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anqnese^ sequence of STofTSSSJS^ 

tS^'^^rilS^^«L^^^^^^^^ in a state . Wc ™. du«. 
proce«:ed sho^be as^S^SS m^S^^JSS*^^ at which units of wotk ate 

ofiteadliaa misses if o^Iitvlevd ^»^^*f 5- level) sauis, tto estpecttd muaber 

4 to quality iBire! q. AiBvennoT? is ri^^^ffla* a!^^^ wMch is pan of state 

quality changefDMtloa, '**^'»'«'«««^fi»«*QnniIiiiisthedH^ 



3.2 MDPjtostaoces 



iiwervals a* chosen, the better modeling of dw^K^wLnf « ^ "»« prosress 

iniHiy fimcdon. diodeadliae xxuss^^a^^^n^I^^ rxetmns. Secondly. y,» have to defioo die 

we need siaiislical dataf \te dBtw» sKsi aSK™ amsilion piobsfl)iUtifis and revenues, 

ifflBaBiitqnaBty tewds for the succesidvft^^ of »n* tr^ ^ and the pmcassiqg times Sot 

s^opprMimaticn, value itenaton ^policy fteS^TS1fS^11lS<^***^^'f'"^ 
ofl^lhie;andiesnltsin ^ optimal J^vprficy/SSwi^ 
qwfity levels, indepettdentof dte fin* of 

ahigheror eqmd qnaiiisr level ch<^ Tf^fi^^^JS^f^lS^ ^ hiijher relative piogiess lesnlts in 

o^«,sr^s!^^ 

have (Dbe sloied. A Maricovpolioy tfaercfowhas a sna^^mS^^^^5^«S^^^'" ttenastOne 



4 Seated Budget Control 

iBla^ ^v«avinpwca<^SSS^^e^^!i^ «nits of wotfc are nor 
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of movie scents. 




It^Dsl. Tl» toad for decerns a seqneoca^MPBC^&WiM SI one qqaRtytev^ 



4J. Load CoinplicatiDxi 

Id HgDte !U [be average load per fiame over liie ^tite sequence ifi about 17^ ms. Rir subsequeace firaake 
0 ^ 100 die average load is about 10 ms, and for sul^eqiience frame 100 300 it ib about 23 ms. If the 
pequencB fe procefised nsiag a budget 17-5 ms, then applying an optimal MarKov policy derived ftwr 
ibssiT^xxtfi wonldte^idtiasuboptiii^Qli^^ the^fexkov 
poUcyTK»ddl)eiDopes^misticJeadi3qgtotowetd^ 
100 30(^ die SidbikOT pcdi<^ TOiddbe too ^ 

One option co talce cfaex^adon in load between sneeeSGive unite of woik into aeootmt is tor eztendifig the 
fiiaiB space of die Ma&ov mod^ with lead complication. We define load campHcanon as die picpordon 
bE^men the stnictnral load and the average load for a given qnaHQr level of processing. We assume Hint Ihe 
load complicaiion is depen dent on the complexiQr of the vaot$ of work that aie processed, an d no t on the 
vsedqiuality level of piiocesshig:. Accoitling to ineasuiaiieat$£or a scalable ^9^C^Z decoder^ tid$ see a 
mt assttmption. Note diat Oze load cos^ioadon id also independent of the >^^ed btidgei. 
The load com^lieadon in snbseisuenee fiame 0 — 100 is about 0^, and in subsequence frame lOO 300 
il is tib(M 1,31* lb be mehtded in the state space, load compHcaiion should just like rcladve progress be 
^ttedzed, Kowev^ aproUmis that die Maricov model sofiTeis fiomthe curse cf dimensionality [ll» 
rifds means diat die nuixibef of states gicowsesponentiai^^ Testate 
space muld become too large fbr the problem to be solvable hi reasonable dme. Another dzaiwbadc is that 
a lot note meaauEe data would be xequiced co dem^ luliable statislics for the problem. So, M$ ^picacb to 
nnxtel tfaetdatioaln load between successive units ofworicis undesirable* 



4.2 BudgetScaUng 

Anodierappzoach Is ID model dte relation in load bernreesisuceo As- 
jnimediatdte sequence is processed using a budgets 7= 20 ms. Insubsequence&ameO^lOO, die load 
compBoadOn is about 0.57. Qearfy, processing at a load complication of 0^7 using budget bi^Mk» pro- 
cessfeg at a. load C0a»plication of but using ^budget that is 0^ times smaller dian its acmal value. So, 
inateadof etpplying m c^pttoalMaitov poH<y derived ftr * w 20 ms, we apply an optimal Markov policy 
derived fl>raodedbttd^2Qnw/0.57« 35 ms. Similatly. in subsequence ficame 100 - 300 we apply an 
OFpdmal hias^v policy derived &r scaled bucteet 20m/l^i « IS njs. We call diis mediod to cornrol die 
qiualiv afppxoee&sSDS^caledbudgetccntrvh 

deaiiyi it js «ot piacdcal ro deiive opdmal Markov policies fas each possible scaledlyudgec. Thecetor^ we 
only tierive opdmal Matov policies fiora. small number of budgets, and imei!p61ate betKreen these policies 
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43 



we assnme (bat the comolieation fiM^<.S . ^W- /nst like toad coajpficanoii 

only lite anSSSST^etSS'^^M.'K IbnutoUlKm 
iWHUgB over tte coo»H<aSfe2s^S5^«* load flnotoitons by eompaSag a mmliiir 

iIu>riinnmgawwHcaaonfl«storfor«ititetf 

becomes. In esse Ot =s 1. oiilv th^ latr J^L ^ rarmgnted the nmning averafift 

care of stfWctURa toad flHitotoMtar^^ Bccansewealreaifytato 
policy. weisStoSwSSKS^tfS^ 
^^^^^^ ^**™^"'^«on'P«s«SOi«yiei)apa 



lut] 
given 



5 Off-line Optfmnm 

6 Simnlatiojii Experiments 

Video sr«»m as inptttftedecoderBeqnonSSSdSS^ 

fiames ooow^ondnitli units of woS. Fram^c^lXniw^^ T v « madenp. We let 

<jWfty fevcIS, in iocieasiqg qJS ffldSS^f^^^*^**^"*" f " be dfecodedta five cKflTenat 
«ms4uamyanifirii»aea^,,...,^j. There ate ilBeedif6asBntQTpes<tfMlffiG.2 

5 
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6^mes: ixitrd cotied (I-ftmies), |tie£ciive coded 0?-ftames). and tnsBtectianal codftd (B-fifixnes). Hie fSme 
reqimed to decode a frame of a giyoi type in a ^eo qtraliiy level is vadalde. Kaoamly depeodeat on fte 

compIeodQT of the frame. 

We use the scalable de<»d^ tx> create traces £i(irMFB&2 coded video dreoms. A tmoe contains 

fix: eaeb fianie ID the video fitieam its type Bxidtbeiitxief^^ 

IteteB serve two poiposes. Hcst; to derive statistics ftat ane usedto conyme transftiftn pgolwbilicie& and 
f^vetmesfor MDP instances. Second, to sinnilaiB ihe execolSotiof ctedecodei; vvliich wedo io cescqnaliiy 
level control stiaiegi^ 

We define tlie pat^e^exs for MPF instances as follows. The upper botmd on reladve piogtess ptuec ^ 
diosen eqaal to 3, v^bich cozresponds to nsing oaqnit buffer wMdi 3 decoded finmes cat be stored. We 
dmose 500 progcass intervals ixibetween arelaUve ^gress of zero and ptnsz, Tbis mxmber of pto&ee$B 
intervals is diexes^ of a txadeoffioiade b^een the sobuion q^aIi1y and ti^ computational eoot0exity of 
tbeMDPinstances, fbrwhich werefcrto [13]. The utility fimctioawfe) is defined by = 1,«(^) =2J5r 
s 5r uiq^) 7 A u{q$) ^ 10. Tte deadline zniss penalty is defined by lOOC^ ivblcfa loi^^ tsmaz 
^bat we allow at most 1 deadfine miss per 100 fcames. The qu^iy diange ftmcdon is defined by a penal^ 
of 0 &r not changing die quality level at a milestone, a peoal^y^ <^4 for ittovinfi 1 qnaE tjr level rip or down, a 
penahy of 7 for moving 2 qual^ levels up or down, a penalty of 9 for moving 3 quaU& leveJs or down, 
end a, penalty of 10 for moving 4 quality levels up or dowit. 

Apart ton fiiese fixed parameters, we also have three pazomete^s that we vary to define di£6^£eat MDP 
insmnoes. Fits^ we vary tbe trace wbose statistics are used to compuie transition probabiBties and revenue?. 
Second^, we can choose wbetb^ or not fiame types should bo taken hno account. Kframe typesB are not 
taken sno account, I-JP- and B-fiames aro ell xsg^ded to be of Ibe same type. To ^tinguish between botti 
cases, we attach a subscripi: ») the name of the liace lO indicate that fiiame are t^ken Into aocomit. 
Flnal^» we also vary budget to define dilferencMDP insiances. 

G^en a trace and a decision about die use of £name iypes» we define a set of 41 different MDP fnscmces by 
varying b ficomS ms to45 ms* using intejmetfiate steps of 1 ms. lb solve tiie set ofMDP instances we apply 
successive ^pioximation u^£ an inaecuiaiCy x)aEameter of 0.0l» This r^iults in a set of optixual l^/fadoov 
poHdfiSr onepoli^ for each budget Next* w^use tiiepohdes to consouct quality level control strategies. If 
tbe decoder processes using a fixed budget tiien we can direct!^ ^ply die optimal Madfiov poliey deziyed 
fbr & to control the quality lev^ at which fcames are decoded. For trace t this control strata is denotedby 
%1PC7). We can also use the set of optimal M&r&ov policies to inkj^eoteatscaled budget conoid. Pbr tRSCO-r 
and st^-si^e parameter a this conirol strategy is deuoiedby SBC(^). 

Tb test a control strategy, we simulate the processing of die deooderu^g the control strategy 41 fimesp'fiir 

6 varying :&om 5 rns to 45 rns using iiusime^ate st^s of 1 ms. In these siomlation we ap 

deadline miss epproach. lb compare GOnu:ol strategies, in each simuiatioa we measure tfaequaliQr lev^ 
usage* die deadline misses and the quality level changes . From these measurements we ooxnpuos fOe each 
simuiationthe average revemeper transition* which is the tbing vto want to optimize. 
We have performed tiuiee ex^perimeats. 

Pterfonnance of Control Stratcgifis 
^ Xheiofluence of frame types 
6^ Uke&flnnweof the trace 

7 Condtii^xis 
laxeriavikQen 

tU B,E» Bellman. Dynamie Programming^ Princeton UnivBtsiiy P^ess, Princeiou, 1957 . 

eg RJ.^Jil.- M- CSabmait a Hcmschel, G.C. vanLoo, andEJ^M. Stefibns, QoS fiarconswmer termiaate 
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tl21 DJ, Widie. Markov DedsicnProceaes. John "Wiley & Sons Ltd.. 1993. 
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1- Method of setting an output quality of a next media firame, wherein 

the coJputguaUty is provided byaaediaprocessmgappHcation; and 
the media piocesaing appiicatxoa is designed to provide a plwality of output 
qualities of ^ next media ^am» 

characterized in that setting the output quaUty of the next media ftame is based upon a self- 
teaming control strategy U,at uses accessing time and an output quality of a previous 
media fiame to decetznine the ouq»ut quality of the next media frame. 

2. System to set an ou^ quality of a next media frame, comptising 

application means conceived to piovi de the ouiput qualify of a pluraHty of 
output qualities of tbe next media ftame 

characterized in Hiat the aystaoa fcrther comprises 
contiDl means concaved to set tbe output quality of the next media ftame. 
based upon a self^leaxning coniiol swi^ that uses a processing time and 
of apieviousmediafiametodetetminetheoutputqualityof the next media feame. 

3. '^«*»°»P»M»P«>Sram product designed to perfoxmiiie method 

20 acccndiitgto claims 1. 



4- '^'^'^^^fi^^^cecompriaiagaconwueerprogtamprodBctaM 
claims. 



25 5. 



A television set compiishig a system according claim 2. 



6. 



A television set comprising a system according <dalm 2. 
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